hotfix(chat): Chat answer streaming and smooth markdown rendering by AnishSarkar22 · Pull Request #1515 · MODSetter/SurfSense

AnishSarkar22 · 2026-06-18T18:36:03Z

Description

Enabled token streaming when constructing chat LLM bundles by passing streaming=True for both DB-backed and global model paths.
Restored smooth assistant response rendering in MarkdownTextPrimitive so streamed answers reveal progressively instead of appearing in rigid chunks.
Added unit coverage for LLM bundle construction to verify streaming is enabled for DB-backed and global models.

Motivation and Context

FIX #

Screenshots

API Changes

This PR includes API changes

Change Type

Testing Performed

Tested locally
Manual/QA verification

Checklist

Follows project coding standards and conventions
Documentation updated as needed
Dependencies updated as needed
No lint/build errors or new warnings
All relevant tests are passing

High-level PR Summary

This hotfix resolves chat answer streaming issues by enabling token streaming in the LLM bundle construction for both database-backed and global model configurations. The backend now explicitly passes streaming=True when creating SanitizedChatLiteLLM instances, fixing the default non-streaming behavior. On the frontend, smooth markdown rendering is restored by re-enabling the smooth property in MarkdownTextPrimitive and restoring auto-scroll behavior in the chat viewport, ensuring streamed responses appear progressively rather than in rigid chunks. The changes include comprehensive unit tests to verify streaming is properly configured for both model loading paths.

⏱️ Estimated Review Time: 5-15 minutes

💡 Review Order Suggestion

Order	File Path
1	`surfsense_backend/tests/unit/tasks/chat/streaming/test_llm_bundle.py`
2	`surfsense_backend/app/tasks/chat/streaming/flows/shared/llm_bundle.py`
3	`surfsense_web/components/assistant-ui/markdown-text.tsx`
4	`surfsense_web/components/assistant-ui/chat-viewport.tsx`

Summary by CodeRabbit

New Features
- Chat viewport now auto-scrolls to bottom when new messages arrive, improving message visibility
- Markdown text rendering now includes smooth transitions for a better visual experience
Bug Fixes
- Fixed streaming configuration to ensure it is correctly applied across all LLM model types
Tests
- Added comprehensive unit tests for streaming LLM bundle loading and configuration

- Introduced a new test file to validate the LLM bundle construction for streaming flows in chat. - Implemented tests to ensure that both DB-backed and global models enable streaming correctly. - Utilized mocking to isolate dependencies and verify the expected behavior of the LLM constructor.

- Updated the LLM bundle construction to include a streaming option for both DB-backed and global models. - Modified the `litellm_kwargs` to set the streaming parameter to True, enhancing the functionality for chat streaming flows.

- Updated the MarkdownTextPrimitive component to enable smooth rendering by default. - Adjusted the props to streamline the rendering process for improved user experience.

vercel · 2026-06-18T18:36:07Z

@AnishSarkar22 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

coderabbitai · 2026-06-18T18:36:42Z

📝 Walkthrough

Walkthrough

load_llm_bundle now explicitly forces streaming=True when constructing SanitizedChatLiteLLM in both the DB-backed and global model paths. New unit tests verify this contract for both paths. In the frontend, ChatViewport enables auto-scroll props and MarkdownTextImpl enables smooth rendering.

Changes

Backend: Force streaming in LLM bundle

Layer / File(s)	Summary
Force `streaming=True` in both LLM bundle paths `surfsense_backend/app/tasks/chat/streaming/flows/shared/llm_bundle.py`	Merges `"streaming": True` into `litellm_kwargs` when constructing `SanitizedChatLiteLLM` for both the `config_id > 0` (DB-backed) and `config_id < 0` (global/virtual) model paths.
Unit tests asserting `streaming=True` for both model paths `surfsense_backend/tests/unit/tasks/chat/streaming/test_llm_bundle.py`	New test module with an autouse fixture that stubs shared dependencies; two async tests assert `load_llm_bundle` passes `streaming=True` plus the correct model identifiers and credentials to `SanitizedChatLiteLLM` for both path variants.

Frontend: Enable auto-scroll and smooth markdown rendering

Layer / File(s)	Summary
Enable viewport auto-scroll and markdown smooth props `surfsense_web/components/assistant-ui/chat-viewport.tsx`, `surfsense_web/components/assistant-ui/markdown-text.tsx`	`autoScroll` and `scrollToBottomOnRunStart` are switched from `false` to enabled in `ChatViewport`; `smooth` is switched from `false` to enabled in `MarkdownTextImpl`.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 A rabbit hops through streams of data flow,
Forcing streaming=True wherever LLMs go.
The viewport now scrolls with a smooth little glide,
And markdown animations no longer hide.
Hop hop hooray, the tests confirm it's right —
Everything streams into the chat just right! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the two main changes: enabling chat answer streaming and smooth markdown rendering, matching the core modifications across the codebase.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

🧹 Nitpick comments (1)

surfsense_backend/tests/unit/tasks/chat/streaming/test_llm_bundle.py (1)

75-78: ⚡ Quick win

Strengthen forced-streaming override coverage.

Line 75 and Line 132 stub kwargs without a streaming field, so these tests don’t prove precedence when upstream provides streaming=False. Seed that value and keep expected streaming=True to lock the override contract.

Proposed test hardening

     monkeypatch.setattr(
         llm_bundle,
         "to_litellm",
         lambda _conn, _model_id: (
             "openai/gpt-4o-mini",
-            {"api_key": "sk-test", "temperature": 0.1},
+            {"api_key": "sk-test", "temperature": 0.1, "streaming": False},
         ),
     )
@@
     monkeypatch.setattr(
         llm_bundle,
         "to_litellm",
         lambda _conn, _model_id: (
             "anthropic/claude-sonnet-4-5",
-            {"api_key": "sk-ant-test", "temperature": 0.2},
+            {"api_key": "sk-ant-test", "temperature": 0.2, "streaming": False},
         ),
     )

Also applies to: 132-135

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@surfsense_backend/tests/unit/tasks/chat/streaming/test_llm_bundle.py` around
lines 75 - 78, The test stubs at line 75 and line 132 are missing a streaming
field in their kwargs dictionaries, which means they don't actually verify the
override behavior when upstream explicitly provides streaming=False. Add
streaming: False to the kwargs dictionary returned by the lambda functions at
both locations, while keeping the expected streaming=True assertion in the test
to confirm the override mechanism properly handles and overrides explicit False
values from upstream.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@surfsense_backend/tests/unit/tasks/chat/streaming/test_llm_bundle.py`:
- Around line 75-78: The test stubs at line 75 and line 132 are missing a
streaming field in their kwargs dictionaries, which means they don't actually
verify the override behavior when upstream explicitly provides streaming=False.
Add streaming: False to the kwargs dictionary returned by the lambda functions
at both locations, while keeping the expected streaming=True assertion in the
test to confirm the override mechanism properly handles and overrides explicit
False values from upstream.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 018339dc-ff0f-4236-ab23-065a88eeab32

📥 Commits

Reviewing files that changed from the base of the PR and between b233f15 and bb664a1.

📒 Files selected for processing (4)

surfsense_backend/app/tasks/chat/streaming/flows/shared/llm_bundle.py
surfsense_backend/tests/unit/tasks/chat/streaming/test_llm_bundle.py
surfsense_web/components/assistant-ui/chat-viewport.tsx
surfsense_web/components/assistant-ui/markdown-text.tsx

AnishSarkar22 added 3 commits June 18, 2026 23:32

feat: enable streaming in LLM bundle construction

6e970be

- Updated the LLM bundle construction to include a streaming option for both DB-backed and global models. - Modified the `litellm_kwargs` to set the streaming parameter to True, enhancing the functionality for chat streaming flows.

fix: enable smooth rendering in MarkdownText component

bb664a1

- Updated the MarkdownTextPrimitive component to enable smooth rendering by default. - Adjusted the props to streamline the rendering process for improved user experience.

AnishSarkar22 changed the title ~~hotfix: Chat answer streaming and smooth markdown rendering~~ hotfix(chat): Chat answer streaming and smooth markdown rendering Jun 18, 2026

coderabbitai Bot reviewed Jun 18, 2026

View reviewed changes

MODSetter merged commit 03e57bd into MODSetter:main Jun 18, 2026
9 of 12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

hotfix(chat): Chat answer streaming and smooth markdown rendering#1515

hotfix(chat): Chat answer streaming and smooth markdown rendering#1515
MODSetter merged 3 commits into
MODSetter:mainfrom
AnishSarkar22:hotfix/streaming

AnishSarkar22 commented Jun 18, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

vercel Bot commented Jun 18, 2026

Uh oh!

coderabbitai Bot commented Jun 18, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Poem

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

AnishSarkar22 commented Jun 18, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Screenshots

API Changes

Change Type

Testing Performed

Checklist

High-level PR Summary

Summary by CodeRabbit

Uh oh!

vercel Bot commented Jun 18, 2026

Uh oh!

coderabbitai Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AnishSarkar22 commented Jun 18, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 18, 2026 •

edited

Loading